New quasi-Newton methods for unconstrained optimization problems

Many methods for solving minimization problems are variants of Newton method, which requires the specification of the Hessian matrix of second derivatives. QuasiNewton methods are intended for the situation where the Hessian is expensive or difficult to calculate. Quasi-Newton methods use only first derivatives to build an approximate Hessian over a number of iterations. This approximation is updated each iteration by a matrix of low rank. In unconstrained minimization, the original quasi-Newton equation is Bk+1sk = yk, where yk is the difference of the gradients at the last two iterates. In this paper, we first propose a new quasi-Newton equation Bkþ1sk 1⁄4 y k in which y k is decided by the sum of yk and Aksk where Ak is some matrix. Then we give two choices of Ak which carry some second order information from the Hessian of the objective 0096-3003/$ see front matter 2005 Elsevier Inc. All rights reserved. doi:10.1016/j.amc.2005.08.027 * Corresponding author. E-mail addresses: [email protected] (Z. Wei), [email protected] (G. Li), maqilq@polyu. edu.hk (L. Qi). 1 The work of this author was done during his visit to the Department of Applied Mathematics, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong. His work is supported by the Croucher Foundation of Hong Kong, Chinese NSF grants 10161002 and Guangxi NSF grants 9811020. 2 The work of this author is supported by the Research Grant Council of Hong Kong. Z. Wei et al. / Appl. Math. Comput. 175 (2006) 1156–1188 1157 function. The three corresponding BFGS-TYPE algorithms are proved to possess global convergence property. The superlinear convergence of the one algorithm is proved. Extensive numerical experiments have been conducted which show that the proposed algorithms are very encouraging. 2005 Elsevier Inc. All rights reserved.